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1 An evaluation of animated demonstrations of learnin g computer-based tasks 
Susan Palmlter, Jay Elkerton 

March 1991 Proceedings of the SIGCHI conference on Human factors in computing 
systems: Reaching through technology 

Full text available: pdf(865.02 KB) Additional Information: full citation , references , citings , index terms 



2 A case for interaction: a study of interactive information retrieval behavior and 
effectiveness 

Jurgen Koenemann, Nicholas J. Belkin 

April 1996 Proceedings of the SIGCHI conference on Human factors in computing 
systems: common ground 

Full text available: ^p^li8M8)„gl Additional Information: full citation, referenc es, citings, index terms 
htmU42.88 KB) 



Keywords: empirical studies, evaluation, infornnation retrieval, relevance feedback, user 
interfaces 



3 Manufacturing applications: Manufacturing 1: simulation-based analysis of a com plex Q 
printed circuit board testin g process 
Jeffrey S. Snnith, Yali Li, Jason Gjesvold 

December 2002 Proceedings of the 34th conference on Winter simulation: exploring 
new frontiers 

Full text available: ^ pdf(1 77.73 KB) Additional Information: full citation , abstract, references 

This paper describes a simulation-based analysis of a printed circuit board (PCB) testing 
process. The PCBs are used In a defense application and the testing process is fairly 
complex. Boards are mounted on a test unit in batches and go through three thermal test 
cycles. As boards fail testing during the thermal cycling, operators can either replace the 
failed boards at fixed points during the cycling or can allow the test unit to complete the 
testing cycle before removing failed boards. The ... 

^ Modeling methodolo g y b: Improving the model development process: model testing: is Q 
it only a special case of software testing ? 
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C. Michael Overstreet 

December 2002 Proceedings of the 34th conference on Winter simulation: exploring 
new frontiers 

Full text available: ^ pdf(201.11 KB) Additional Information: full citation , abstract , references 

Effective testing of software is an important concern in the software engineering 
community. While many techniques regularly used for testing software apply equally well to 
testing the implementations of simulation models, we believe that testing simulations often 
raises issues that occur infrequently in other types of software. We believe that many code 
characteristics that commonly occur in simulation code are precisely those that the software 
testing community has identified as making test ... 

5 Improvin g student p erformance by evaluating how well students test their own | 
programs 
Stephen H. Edwards 

September 2003 Journal on Educational Resources in Computing (JERIC), volumes issues 
Full text available: ^ p d f (347.57 KB ) Additional Information: full citation , abstract , references , index terms 

Students need to learn more software testing skills. This paper presents an approach to 
teaching software testing in a way that will encourage students to practice testing skills in 
many classes and give them concrete feedback on their testing performance, without 
requiring a new course, any new faculty resources, or a significant number of lecture hours 
in each course where testing will be practiced. The strategy Is to give students basic 
exposure to test-driven development, and then provide ... 

Keywords: agile methods, extreme programming, teaching software testing, test-driven 
development, test-first coding 



6 On Connpacting Test Response Data Containing Unknown Values Q 
Chen Wang, Sudhakar M. Reddy, Irith Pomeranz, Janusz Rajski, Jerzy Tyszer 

November 2003 Proceedings of the 2003 IEEE/ACM international conference on 
Computer-aided design 

Full text available: ^pdf(1 80.56 KB) Additional Information: full citation , abstract 

The design of a test response compactor called a Block Compactoris given. Block 
Compactors belong to a new class of compactorscalled Finite Memory Compactors. Different 
from spacecompactors, finite memory compactors contain memory elements. Also unlike 
time compactors, finite memory compactors havefinite impulse response. These properties 
give finite memorycompactors the ability to achieve higher compaction ratios thanspace 
compactors and still be able to tolerate unknown values intest responses. ... 

7 Hardware/Software C o-testing of Embedded Memories in Complex SOCs Q 
Bai Hong Fang, Qiang Xu, Nicoia Nicolici 

November 2003 Proceedings of the 2003 IEEE/ACM international conference on 
Computer-aided design 

Full text available: ^ pdf(1 45.29 KB) Additional Information: full citation , abstract 

A novel approach for testing embedded memories in complexsystems-on-a-chip (SOCs) is 
presented. The proposedsolution aims to balance the usage of the existing on-chipresources 
and dedicated design for test (DFT) hardwaresuch that the functional power constraints are 
not exceededduring test while trading-off the testing time agalnstDFT area and 
performance overhead. The suitability ofsoftwa re-centric and hardware-centric approaches 
for embeddedmemory testing is examined and to combine the advanta ... 
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Scheduling with Power Constraints 

Yu Xia, Malgorzata Chrzanowska-Jeske, Benyi Wang, Marcin Jeske 
November 2003 Proceedings of the 2003 IEEE/ACM international conference on 
Computer-aided design 

Full text available: ^ pdf(265.05 KB) Additional Information: full citation , abstract 

We present a new algorithm to co-optinnize test scheduling andwrapper design under power 
constraints for core-based SoCs(System on Chip). Core testing solutions are generated as a 
set ofwrapper designs, each represented by a rectangle with width equalto the test time 
and height equal to the number of TAM (TestAccess Mechanism) wires used. The test- 
scheduling problem withpower constraints is formulated as the distributed rectangle bin- 
packingproblem, which allows wrapper pins to be assigned to non ... 

^ TAM Optimization for Mixed-Signal SOCs using Analog Test Wrappers Q 
Anuja Sehgal, Sule Ozev, Krishnendu Chakrabarty 

November 2003 Proceedings of the 2003 IEEE/ACM international conference on 
Computer-aided design 

Full text available: ^ pdfd 67.79 KB) Additional Information: full citation , abstract 

We present a new approach for TAM optimization and testscheduling in the modular testing 
of mixed-signal SOCs. A testplanning approach for digital SOCs is extended to handle 
analogcores in a plug-and-play fashion. A test wrapper based on anADC/DAC pair and a 
digital configuration circuit is designed foranalog cores such that these cores can be 
accessed through digitalTAMs. In this way, there is no dependence on an analog testbus 
and expensive mixed-signal testers. Experimental results arepresent ... 

^0 Partial Core Encryption for Performance-Efficient Test of SOCs 
Ozgur Sinanoglu, Alex Orailoglu 

November 2003 Proceedings of the 2003 IEEE/ACM international conference on 
Computer-aided design 

Full text available: ^ pdf(1 40.35 KB) Additional Infomiation: full citation , abstract 

The isolation of a core through full I/O scan helps ease SOC test challenges;yet the 
performance of high-speed SOCs is significantly hampered. We propose a partial core 
encryption methodology wherein thecore vendor unveils only a small part of the core logic, 
successfully satisfyingcore IP protection requirements. Once the partially encryptedcores 
are merged into an SOC, the system integrator performs test generationon the visible SOC 
logic only, greatly reducing the test generationeffort expende ... 

11 Experience reports: testing and fault correction: Requirements discovery during the 

testing of safety-critical software 
Robyn R. Lutz, Ines Carmen Mikulski 

May 2003 Proceedings of tlie 25th International Conference on Software Engineering 

Full text available:^ pdf(650.94 KB) 

Additional Information: full citation , abstract , references 

^ Publisher Site 

This paper describes the role of requirements discovery during the testing of a safety- 
critical software system. Analysis of problem reports generated by the integration and 
system testing of an embedded, safety-critical software system identified four common 
mechanisms for requirements discovery and resolution during testing: (1) Incomplete 
requirements, resolved by changes to the software, (2) Unexpected requirements 
interactions, resolved by changes to the operational procedures, (3) Requirem ... 

12 Experience reports: testing and fault correction: Assessing test-driven development at Q 
IBM 

E. Michael Maximilien, Laurie Williams 
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May 2003 Proceedings of the 25th International Conference on Software Engineering 

Full text available: Ipa pdf(572.29 KB) 

J^"^ Additional Information: full citation , abstract , references , citings 

W Publisher Site 

In a software developnnent group of IBM Retail Store Solutions, we built a non-trivial 
software systenn based on a stable standard specification using a disciplined, rigorous unit 
testing and build approach based on the test- driven developnnent (TDD) practice. Using 
this practice, we reduced our defect rate by about 50 percent compared to a similar system 
that was built using an ad-hoc unit testing approach. The project completed on time with 
minimal development productivity impact. Additionally, ... 

13 Technical papers: testing II: Data flow testing as nnodel checking 
Hyoung Seok Hong, Sung Deok Cha, Insup Lee, Oleg Sokolsky, Hasan Ural 

May 2003 Proceedings of the 25th International Conference on Software Engineering 

Full text available: ^ . rvrxn *ov iSI 

T^„pdi(1.00 MBj'aEP' Additional Information: full citation , abstract , references 

Publisher Site 

This paper presents a model checking-based approach to data flow testing. We characterize 
data flow oriented coverage criteria in temporal logic such that the problem of test 
generation is reduced to the problem of finding witnesses for a set of temporal logic 
formulas. The capability of model checkers to construct witnesses and counterexamples 
allows test generation to be fully automatic. We discuss complexity issues in minimal cost 
test generation and describe heurstic test generation algorith ... 

14 Technical papers: testing II: A framework for component deploynnent testing 
Antonia Bertolino, Andrea Polini 

May 2003 Proceedings of the 25th International Conference on Software Engineering 

Full text available:. 



^pdf(1.34 MB) Iff Additional Information: full citation , abstract , references 
Publisher Site 

Component-based development is the emerging paradigm in software production, though 
several challenges still slow down its full taking up. In particular, the "component trust 
problem" refers to how adequate guarantees and documentation about a component' s 
behaviour can be transferred from the component developer to its potential users. The 
capability to test a component when deployed within the target application environment can 
help establish the compliance of a candidate component to the cust ... 

1 5 Technical papers: testing II: Fragment class analysis for testing of polymorphism in Q 
Java software 

Atanas Rountev, Ana Milanova, Barbara G. Ryder 

May 2003 Proceedings of the 25th International Conference on Software Engineering 

Full text available: 



|pdf(1.13 MB)^ Additional Information: full citation , abstract , references , citings 
Publisher Site 

Adequate testing of polymorphism in object-oriented software requires coverage of all 
possible bindings of receiver classes and target methods at call sites. Tools that measure 
this coverage need to use class analysis to compute the coverage requirements. However, 
traditional whole-program class analysis cannot be used when testing partial programs. To 
solve this problem, we present a general approach for adapting whole-program class 
analyses to operate on program fragments. Furthermore, ... 

^6 Technical papers: testing I: Improving test suites via operational abstraction 
Michael Harder, Jeff Mellen, Michael D. Ernst 

May 2003 Proceedings of the 25th International Conference on Software Engineering 
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Full text available: ^ . iSB Additional Information: full citation , abstract , references , citings 

^ pdf(1.31 MB) ,^ 

Publisher Site 

This paper presents the operational difference technique for generating, augnnenting, and 
minimizing test suites. The technique is analogous to structural code coverage techniques, 
but it operates in the semantic domain of program properties rather than the syntactic 
domain of program text.The operational difference technique automatically selects test 
cases; it assumes only the existence of a source of test cases. The technique dynamically 
generates operational abstractions (which describe obser ... 

^ ^ Technical papers: testing I: Improving web application testing with user session data Q 
Sebastian Elbaum, Srikanth Karre, Gregg Rothermel 

May 2003 Proceedings of the 25th International Conference on Software Engineering 

Full text available:^ ^r/^ -iVm^^iox [§1 

■^^pduLig MB),^ Additional Information: full citation , abstract , references 

Publisher Site 

Web applications have become critical components of the global information infrastructure, 
and it is important that they be validated to ensure their reliability. Therefore, many 
techniques and tools for validating web applications have been created. Only a few of these 
techniques, however, have addressed problems of testing the functionality of web 
applications, and those that do have not fully considered the unique attributes of web 
applications. In this paper we explore the notion that user s ... 

^ Technical pa pers: testin g I: Constructing test suites for interaction testing Q 
Myra B. Cohen, Peter B. Gibbons, Warwick B. Mugridge, Charles J. Colbourn 
May 2003 Proceedings of the 25th International Conference on Software Engineering 

Full text available: « pdf(975.28 KB ) 

M Additional Information: full citation , abstract , references , citings 

ffl P Publisher Site 

Software system faults are often caused by unexpected interactions among components. 
Yet the size of a test suite required to test all possible combinations of interactions can be 
prohibitive in even a moderately sized project. Instead, we may use pairwise or t-way 
testing to provide a guarantee that all pairs or t-way combinations of components are 
tested together. This concept draws on methods used in statistical testing for 
manufacturing and has been extended to software system testing. A cov ... 

19 Technical papers: program analysis: Visualization of test information to assist fault Q 
localization 

James A. Jones, Mary Jean Harrold, John Stasko 

May 2002 Proceedings of the 24th International Conference on Software Engineering 

r- MX ^ I ui 01 ^r/. ^cK^Dx Additional Information: full citation , abstract , references , citin g s , index 

Full text available: TO pdf(1.45 MB) ^ ^ 

^ terms 

One of the most expensive and time-consuming components of the debugging process is 
locating the errors or faults. To locate faults, developers must identify statements involved 
In failures and select suspicious statements that might contain faults. This paper presents a 
new technique that uses visualization to assist with these tasks. The technique uses color to 
visually map the participation of each program statement in the outcome of the execution 
of the program with a test suite, consisting ... 

20 Technical pa pers: software testin g : Autonnated test case generation for spreadsheets Q 
Marc Fisher, Mingming Cao, Gregg Rothermel, Curtis R. Cook, Margaret M. Burnett 

May 2002 Proceedings of the 24th International Conference on Software Engineering 

.- ^ •■ ui 01 ^f/H >i-7 ^;lD\ Additional Information: full citation , abstract , references , citings , index 

Full text available: 133 pdf (1.47 MB) 

^.t^x terms 
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Spreadsheet languages, which include commercial spreadsheets and various research 
systems, have had a substantial impact on end-user computing. Research shows, however, 
that spreadsheets often contain faults. Thus, in previous work, we presented a methodology 
that assists spreadsheet users in testing their spreadsheet formulas. Our empirical studies 
have shown that this methodology can help end-users test spreadsheets more adequately 
and efficiently; however, the process of generating test cases ... 
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21 Technical papers: software testing: The impact of test suite granularity on the cost- Q 
effectiveness of regression testing 

Gregg Rothermel, Sebastian Elbaum, Alexey Malishevsky, Praveen Kallakuri, Brian Davia 
May 2002 Proceedings of the 24th International Conference on Software Engineering 

Full text available: ^ pdf(1.37 MB) Additional Information: full citation , abstract , references , index terms 

Regression testing is an expensive testing process used to validate software following 
modifications. The cost-effectiveness of regression testing techniques varies with 
characteristics of test suites. One such characteristic, test suite granularity, involves the 
way in which test inputs are grouped into test cases within a test suite. Various cost- 
benefits tradeoffs have been attributed to choices of test suite granularity, but almost no 
research has formally examined these tradeoffs. To addres ... 

22 Technical papers: software testin g : A history-based test prioritization technique for Q 
re gression testing in resource constrained environments 

Jung-Min Kim, Adam Porter 

May 2002 Proceedings of the 24th International Conference on Software Engineering 

Additional Information: full citation , abstract , references , citings, index 
terms 



Full text available: ■g pdf(1.14 MB) 



Regression testing is an expensive and frequently executed maintenance process used to 
revalidate modified software. To Improve it, regression test selection (RTS) techniques 
strive to lower costs without overly reducing effectiveness by carefully selecting a subset of 
the test suite. Under certain conditions, some can even guarantee that the selected test 
cases perform no worse than the original test suite. But this ignores certain software 
development realities such as resource and time constra ... 

ICeywords: empirical study, prioritization, regression testing, test history 



23 Incorporatin g varying test costs and fault severities into test case prioritization 
Sebastian Elbaum, Alexey Malishevsky, Gregg Rothermel 

July 2001 Proceedings of the 23rd International Conference on Software Engineering 

Full text available: S_pdf ( 885.24 KB) Additional Information: full citation , abstract , references , citings , index 
W Publisher Site 

Test case prioritization tecfiniques schedule test cases for regression testing in an order 
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that increases their ability to meet some performance goal. One performance goal, rate of 
fault detection, measures how quickly faults are detected within the testing process. In 
previous work we provided a metric, APFD, for measuring rate of fault detection, and 
techniques for prioritizing test cases to improve APFD, and reported the results of 
experiments using those techniques. This met ... 

Keywords: fault severity, rate of fault detection, regression testing, test case prioritization, 
test cost 



24 An explorative iournev from architectural tests definition down to code tests execution 
Antonla Bertolino, Paola Inverardi, Henry Muccini 

July 2001 Proceedings of the 23rd International Conference on Software Engineering 

Full text available: ^ pdf(496.50 KB) Additional Information: full citation , abstract , references , citings , index 
W Publisher Site 

Our research deals with the use of the Software Architecture (SA) as a reference model for 
the conformance testing of the implemented system with respect to its architectural 
specification, at the integration test level. Having formerly Identified an approach to derive 
architectural test plans, we investigate here the practical meaning of a high level test case 
defined in terms of architectural processes and messages, such as the ones derived by our 
approach. Indeed, establishing a reiatio ... 

25 The specification and testing of quantified progress properties in distributed systems 
Prakash Krishnamurthy, Paolo A. G. Sivilotti 

July 2001 Proceedings of the 23rd International Conference on Software Engineering 

Full text available: g„[>df( 149.64 KB) Additional Information: full citation , abstract , references , citings , index 
W Publisher Site 

There are two basic parts to the behavioral specification of distributed systems: safety and 
progress. In earlier work, we developed a tool to monitor progress properties ofCORBA 
components specified using the temporal operator transient. In this paper, we address the 
specification and testing of transient properties that are quantified (over both bounded and 
unbounded domains). 

We categorize typical quantifications that arise in practical systems and discuss possibi ... 

2* Analysis and testing of Web applications 
Filippo Rlcca, Paolo Tonella 

July 2001 Proceedings of the 23rd International Conference on Software Engineering 

Full text available: ^j pdf(1 67.58 KB ) Additional Information: full citation, abstract , references , citings , index 
W Publisher Site 

The economic relevance of Web applications increases the Importance of controlling and 
improving their quality. Moreover, the new available technologies for their development 
allow the insertion of sophisticated functions, but often leave the developers responsible for 
their organization and evolution. As a consequence, a high demand is emerging for 
methodologies and tools for quality assurance of Web based systems. 

In this paper, a UI^L model of Web applications is proposed for their ... 

Keywords: UML modeling, code analysis, reverse engineering, testing, web applications 
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Testin g : a road map Q 
Mary Jean Harrold 

May 2000 Proceedings of the Conference on The Future of Software Engineering 

Full text available: ^ pdf(1.19 MB) Additional Information: full citation , references , citings , index terms 



28 Network intrusions: Testing network-based intrusion detection signatures using mutant Q 
ex ploits 

Giovanni Vigna, William Robertson, Davide Balzarotti 

October 2004 Proceedings of the 11th ACM conference on Computer and 
communications security 

Full text available:^ pdf(1 97.42 KB) Additional Information: full citation , abstract , references , index terms 

Misuse-based intrusion detection systems rely on models of attacks to identify the 
manifestation of intrusive behavior. Therefore, the ability of these systems to reliably detect 
attacks is strongly affected by the quality of their models, which are often called 
"signatures." A perfect model would be able to detect all the instances of an attack without 
making mistakes, that is, it would produce a 100% detection rate with 0 false alarms. 
Unfortunately, writing good models (or good signatures) ... 

Keywords: intrusion detection, quality metrics, security testing 



29 TEstin g and reliability: Using an SQL covera g e measurement for testing database Q 
ap plications 

Maria Jose Suarez-Cabal, Javier Tuya 

October 2004 Proceedings of the 12tli ACI^ SIGSOFT twelftli international symposium 
on Foundations of software engineering 

Full text available: ^ pdf(256.74 KB) Additional Information: full citation , abstract , references , index terms 

Many software applications have a component based on database management systems in 
which information is generally handled through SQL queries embedded in the application 
code. When automation of software testing is mentioned in the research, this is normally 
associated with programs written in imperative and structured languages. However, the 
problem of automated software testing applied to programs that manage databases using 
SQL is still an open issue. This paper presents a measurement of ... 

Keywords: SQL testing, database testing, software testing, statement coverage, 
verification and validation 



3° TEs ting and reliability: Scalin g r egression testing to large software systems Q 
Alessandro Orso, Nanjuan Shi, Mary Jean Harrold 

October 2004 Proceedings of the 12tli ACM SIGSOFT twelftli international symposium 
on Foundations of software engineering 

Full text available: ^ pdf(303.42 KB) Additional Information: full citation , abstract , references , index terms 

When software is modified, during development and maintenance, it is <i>regression 
tested</i> to provide confidence that the changes did not introduce unexpected errors and 
that new features behave as expected. One important problem in regression testing is how 
to select a subset of test cases, from the test suite used for the original version of the 
software, when testing a modified version of the software. Regression-test-selection 
techniques address this problem. Safe regression-te ... 

Keywords: regression testing, software evolution, software maintenance, test selection, 
testing 
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31 Laws and applications: Privacy preserving database application testing 
Xintao Wu, Yongge Wang, Yuliang Zheng 

October 2003 Proceedings of the 2003 ACM workshop on Privacy in the electronic 
society 

Full text available: ^ pdf( 175.62 KB) Additional Information: full citation , abstract , references , index terms 

Traditionally, application software developers carry out their tests on their own local 
development databases. However, such local databases usually have only a small nunnber 
of sample data and hence cannot simulate satisfactorily a live environment, especially in 
terms of performance and scalability testing. On the other hand, the idea of testing 
applications over live production databases is increasingly problematic in most situations 
primarily due to the fact that such use of liv ... 

Keywords: database application testing, indistinguishability, privacy 



32 Use case-based testing of product lines 
Antonia Bertolino, Stefania Gnesi 

September 2003 ACM SIGSOFT Software Engineering Notes , Proceedings of the 9th 

European software engineering conference held jointly with 11th ACM 
SIGSOFT international symposium on Foundations of software 
engineering, volume 28 issue 5 

Full text available- S pdf(2 02 48 KB) Additional Information: full citation , abstract , references , citings , index 

: terms 

This paper presents PLUTO, a simple and intuitive methodology to manage the testing 
process of product lines, described as Product Lines Use Cases (PLUCs). PLUCs are an 
extension of the well-known Cockburn's Use Cases, a notation based on natural language 
descriptions of requirements. The proposed test methodology is based on the Category 
Partition method, and can be used to derive a generic Test Specification for the product 
line, and a set of relevant test scenarios for a customer specific app ... 

Keywords: black box testing, product lines, use cases 



Leveraging field data for Impact analysis and regression testing 
Alessandro Orso, Taweesup Apiwattanapong, Mary Jean Harrold 

September 2003 ACM SIGSOFT Software Engineering Notes , Proceedings of the 9th 

European software engineering conference held jointly with 11th ACM 

SIGSOFT international symposium on Foundations of software 

engineering, volume 28 issue 5 

I- II * ^ -I ui risit w*/ono i^Dv Additional Information: full citation , abstract , references , citings , index 
Full text available: T3a paT(292.94 KB) ; 

Software products are often released with missing functionality, errors, or incompatibilities 
that may result in failures, inferior performances, or user dissatisfaction. In previous work, 
we presented the Gamma approach, which facilitates remote analysis and measurement of 
deployed software and permits gathering of program-execution data from the field. In this 
paper, we investigate the use of the Gamma approach to support and improve two 
fundamental tasks performed by software engineers ... 

Keywords: gamma technology, impact analysis, regression testing, software engineering 

34 

Regression testing of GUIs 



http://portal.acm.org/results.cfo?query=%22correlation%20between%20traim^ 11/28/04 



Results (page 2): "correlation between training" and test 



Page 5 of? 



Atif M. Memon, Mary Lou Soffa 

September 2003 ACM SIGSOFT Software Engineering Notes , Proceedings of the 9th 

European software engineering conference held jointly with 11th ACM 
SIGSOFT international symposium on Foundations of software 
engineering, volume 28 issue 5 

Full text available: ^ pdf(297.68 KB) Additional Information: full citation , abstract , references , index terms 

Although graphical user interfaces (GUIs) constitute a large part of the software being 
developed today and are typically created using rapid prototyping, there are no effective 
regression testing techniques for GUIs. The needs of GUI regression testing differ from 
those of traditional software. When the structure of a GUI is modified, test cases from the 
original GUI are either reusable or unusable on the modified GUI. Since GUI test case 
generation is expensive, our goal is to make the unusabi ... 

Keywords: GUI call-graph, GUI control-flow graph, GUI testing, call-tree, classification of 
events, regression testing, repairing test cases 

35 Consistency techniques for interprocedural test data generation Q 
Nguyen Tran Sy, Yves Deville 

September 2003 ACM SIGSOFT Software Engineering Notes , Proceedings of the 9th 

European software engineering conference held jointly with 11th ACM 
SIGSOFT international symposium on Foundations of software 
engineering, volume 28 issue 5 

Full text available: ^ pdf(277.04 KB) Additional Information: full citation , abstract , references , index terms 

This paper presents a novel approach for automated test data generation of imperative 
programs containing integer, boolean and/or //oat variables. It extends our previous work 
to programs with procedure calls and arrays. A test program (with procedure calls) is 
represented by an Interprocedural Control Flow Graph (ICFG), The classical testing criteria 
(statement, branch, and path coverage), widely used in unit testing, are extended to the 
ICFG. For path coverage, the specif i ... 

Keywords: arrays, consistency, constraint satisfaction, procedures, software testing, test 
data generation 



36 A family of test adequacy criteria for database-driven applications 
Gregory M. Kapfhammer, Mary Lou Soffa 

September 2003 ACM SIGSOFT Software Engineering Notes , Proceedings of the 9th 

European software engineering conference held jointly with 11th ACM 
SIGSOFT international symposium on Foundations of software 
engineering, volume 28 issue 5 

Full text available: ^ pdf(264.38 KB) Additional Information: full citation , abstract , references , index terms 

Although a software application always executes within a particular environment, current 
testing methods have largely ignored these environmental factors, i^any applications 
execute in an environment that contains a database. In this paper, we propose a family of 
test adequacy criteria that can be used to assess the quality of test suites for database- 
driven applications. Our test adequacy criteria use dataflow information that is associated 
with the entities in a relational database. Furthermore ... 

Keywords: database-driven applications, test adequacy criteria 
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July 1976 ACM SIGSIM Simulation Digest , Proceedings of tlie 4th symposium on 

Simulation of computer systems, volume 7 issue 4 
Full text available: ^ pdf(668.99 KB) Additional Information: full citation , abstract , references , index terms 

Simulation is proving to be a valuable technique in testing connputer progranns. By 
sinnulating different aspects of a program's execution and structure it is possible to detect 
errors and sometimes demonstrate the absence of certain errors in the program. This 
presentation will explore three popular testing methodologies which employ simulation 
techniques. Each methodology is based upon a different type of simulation of the program. 
The differences in error detection capability resulting from ... 

38 Assessing and Innproving Testing of Real-time Software using Simulation Q 
Dr. Joel Henry, Josh C. Stiff, Aric J Shirar 

March 2003 Proceedings of the 36th annual symposium on Simulation 

Full text available: pdfM 07.66 KB) 

M Additional Information: full citation , abstract , index terms 

W Publisiier Site 

Simulation testing of real-time software provides greatadvantages in cost savings, early 
error detection, andrequirements clarification prior to development andhardware/software 
integration testing. However,simulation testing must exercise the software in ways 
thataccurately represent the deployment environment andcover a broad range of 
operational scenarios. In thispaper we will describe a method for developing 
simulationtests, evaluating simulation test results, and utilizing thetest evaluatio ... 

39 Student research competition: Automatic identification of common and special object- Q 

oriented unit tests 
Tao Xie 

October 2004 Companion to the 19th annual ACM SIGPLAN conference on Object- 
oriented programming systems, languages, and applications 

Full text available: ^ pdf(133.14 KB) Additional Infomiation: full citation , abstract , references , index terms 

Common and special test inputs can be created to exercise some common and special 
behavior of the class under test, respectively. Although manually created tests are valuable, 
programmers often overlook some special test inputs. If programmers write down 
specifications, special or common tests can be automatically generated and selected by 
tools. However, specifications are not commonly written in practice. This research develops 
a novel approach for automatically identifying common and sped ... 

Keywords: dynamic Inference, test selection 



Leaming: Here, there, anywhere: remote usability testin g that works Q 
Katherine E. Thompson, Evelyn P. Rozanski, Anne R. Haake 

October 2004 Proceedings of the 5th conference on Information technology education 

Full text available: ^ pdf(204.63 KB) Additional Information: full citation , abstract , references , index terms 

Usability testing is regarded as a key element in user-centered design. Several studies from 
the Standish Group have shown that usability testing increases the chance of developing 
usable software. Companies are faced with many challenges: their customers demand 
usable products at reasonable costs and the customer base is distributed and diverse. 
Unfortunately, usability testing is often perceived as impractical due to the remote and 
distributed location of users, limited access to representa ... 

Keywords: human-computer interaction, remote usability testing, usability testing 
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41 Automated Support for Developnnent. Maintenance, and Testing in the Presence of Q 
Implicit Control Flow 

May 2004 Proceedings of the 26th International Conference on Software Engineering 

Full text available: fiQ pdfd 57.61 KB) 

Additional Information: full citation , abstract 

W Publisher Site 

Although object-oriented languages can improve programmingpractices, their 
characteristics may introducenew problems for software engineers. One important 
problemis the presence of implicit control flow caused byexception handling and 
polymorphism. Implicit controlflow causes complex interactions, and can thus 
complicatesoftware-engineering tasks. To address this problem, wepresent a systematic 
and structured approach, for supportingthese tasks, based on the static and dynamic 
analyses ofconstru ... 

42 Generating Tests from Counterexamples Q 
May 2004 Proceedings of the 26th International Conference on Software Engineering 

Full text available: pdf(263.07 KB) 
W Publisher Site 



Additional Information: full citation , abstract 



We have extended the software model checker BLAST toautomatically generate test suites 
that guarantee full coveragewith respect to a given predicate. More precisely, givena C 
program and a target predicate p, BLAST determinesthe set L of program locations which 
program execution canreach with p true, and automatically generates a set of testvectors 
that exhibit the truth of p at all locations in L Wehave used BLAST to generate test suites 
and to detect deadcode in C programs with up to 30 K lin ... 

43 Bi-Criteria Models for AIMIses Test Suite Reduction Q 
May 2004 Proceedings of the 26th International Conference on Software Engineering 

Full text available:^ pdf(1 88.75 KB) 
Publisher Site 



Additional Information: full citation , abstract 



Using bi-criteria decision making analysis, a new modelfor test suite minimization has been 
developed that pursuestwo objectives: minimizing a test suite with regard to a 
particularlevel of coverage while simultaneously maximizingerror detection rates. This new 
representation makes it possibleto achieve significant reductions in test suite size 
withoutexperiencing a decrease in error detection rates. Usingthe all-uses interprocedural 
data flow testing criterion, twobinary integer linear programm ... 
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44 Automated Generation of Test Programs from Closed Specifications of Classes and Q 
Test Cases 

May 2004 Proceedings of the 26th International Conference on Software Engineering 

Full text available: P pdf(315.07 KB) 

Additional Information: full citation , abstract 

^ Publisher Site 

Most research on automated specification-based softwaretesting has focused on the 
automated generation oftest cases. Before a software system can be tested, it must beset 
up according to the input requirements of the test cases.This setup process is usually 
performed manually, especiallywhen testing complex data structures and databases.After 
the system is properly set up, a test execution tool runsthe system according to the test 
cases and pre-recorded testscripts to obtain the outputs, which a ... 

*5 Using Simulation to Empirically Investigate Test Coverage Criteria Based on Statechart Q 
IVIay 2004 Proceedings of the 26th International Conference on Software Engineering 

Full text available: fa pdf(1 66. 61 KB) 

Additional Information: full citation , abstract 

^ Publisher Site 

A number of testing strategies have been proposedusing state machines and statecharts as 
test models inorder to derive test sequences and validate classes orclass clusters. Though 
such criteria have the advantage ofbeing systematic, little is known on how cost effective 
theyare and how they compare to each other.This article presents a precise simulation and 
analysisprocedure to analyze the cost-effectiveness of statechart-basedtesting techniques. 
We then investigate, using thisprocedure, the co ... 

46 Using test suites in evaluation of machine translation systems Q 
Margaret King, Kirsten Falkedal 

August 1990 Proceedings of the 13th conference on Computational linguistics - Volume 
2 

Full text available: ^ pdf(536.47 KB) Additional Information: full citation , references , citings 



47 Testin g and coverage: Mock object creation for test factorin g Q 
David Saff, Michael D. Ernst 

June 2004 Proceedings of the ACM-SIGPLAN-SIGSOFT workshop on Program analysis 
for software tools and engineering 

Full text available: Q pdf(35.39 KB) Additional Information: full citation , abstract , references , index terms 

Test factoring creates fast, focused unit tests from slow system-wide tests; each new unit 
test exercises only a subset of the functionality exercised by the system tests. Augmenting 
a test suite with factored unit tests, and prioritizing the tests, should catch errors earlier in 
a test run. One way to factor a test is to introduce mock objects. If a test exercises a 
component A, which is designed to issue queries against or mutate another component B, 
the implementation of B can be ... 



Keywords: mock objects, test factoring, unit testing 



48 New scan-based test techniques: Combinin g dictionary coding and LFSR reseeding for Q 

test data compression 

Xiaoyun Sun, Larry Kinney, Bapiraju Vinnakota 

June 2004 Proceedings of the 41st annual conference on Design automation 
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In this paper we describe a method to combine dictionary coding and partial LFSR reseeding 
to improve the ompression efficiency for test data compression. We also present a fast 
matrix calculation method which significantly reduces the computation time to find a 
solution for partial LFSR reseeding. Experimental results on ISCAS89 benchmark circuits 
show that our approach is better than either dictionary coding or LFSR reseeding, and 
outperforms several test data compression methods proposed rece ... 

Keywords: VLSI test, built-in self test 

49 New scan-based test techniques: Scan-BIST based on transition probabilities Q 
> Irith Pomeranz 

June 2004 Proceedings of the 41st annual conference on Design automation 

Full text available: ^ pdf(56.90 KB) Additional Information: full citation , abstract , references , index terms 

We demonstrate that it is possible to generate a deterministic test set that detects all the 
detectable single stuck-at faults in a full-scan circuit such that each test contains a small 
number of transitions from 0 to 1 or from 1 to 0 when considering consecutive input values. 
Using this result we show that built-in test-pattern generation for scan circuits can be based 
on transition probabilities instead of probabilities of specific bits In the test set being 0 or 1. 
The resulting approach ass ... 

Keywords: built-in self-test, scan design 



50 New scan-based test techniques: Scalable selector architecture for x-tolerant 
deternninistic BIST 

Peter Wohl, John A. Waicukauski, Sanjay Patel 

June 2004 Proceedings of the 41st annual conference on Design automation 

Full text available: ^ pdf(237.66 KB) Additional Information: full citation , abstract , references . Index tenns 

X-tolerant deternninistic BIST (XDBIST) was recently presented as a method to efficiently 
compress and apply scan patterns generated by automatic test pattern generation (ATPG) 
in a logic built-in self-test architecture. In this paper we Introduce a novel selector 
architecture that allows arbitrary compression ratios, scales to any number of scan chains 
and minimizes area overhead. XDBIST test-coverage, full X-tolerance and scan-based 
diagnosis ability are preserved and are the same as determlnis ... 

Keywords: test-data compression, test-generation (ATPG) 

51 New scan-based test techniques: On the generation of scan-based test sets with Q 
reachable states for testing under functional operation conditions 

Irith Pomeranz 

June 2004 Proceedings of the 41st annual conference on Design automation 

Full text available: ^pdff 120.42 KB) Additional Information: full citation , abstract , references , index terms 

Design-for-testability (DFT) for synchronous sequential circuits allows the generation and 
application of tests that rely on non-functional operation of the circuit. This can result in 
unnecessary yield loss due to the detection of faults that do not affect normal circuit 
operation. Considering single stuck-at faults in full-scan circuits, a test vector consists of a 
primary input vector U and a state S .We say that the test vector consisting of U and S 
relies ... 

Keywords: functional tests, reachable states, scan design 
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52 Advanced test solutions: On test generation for transition faults with minimized peak Q 
power dissipation 

Wei Li, Sudhakar M. Reddy, Irith Pomeranz 

June 2004 Proceedings of the 41st annual conference on Design automation 



This paper presents a method of generating tests for transition faults using tests for stucl<- 
at faults such that the peak power is the minimum possible using a given set of tests for 
stuck-at faults. The proposed method is suitable for use in testing scan designs that employ 
enhanced scan. The method reduces the peak power consumption in benchmark circuits by 
19% on tlie average with essentially the same test set size and the same fault coverage 
compared to an earlier method. 

Keywords: power dissipation, test generation, transition faults 

53 Advanced test solutions: Efficient on-line testing of FPGAs with provable Q 
diagnosabilities 

Vinay Verma, Shantanu Dutt, Vishal Suthar 

June 2004 Proceedings of the 41st annual conference on Design automation 

Full text available: ^ pdf (335.66 KB) Additional Information: full citation , abstract , references , index terms 

We present novel and efficient methods for on-line testing In FPGAs. The testing approach 
uses a ROving TEster (ROTE), which has provable diagnosabilities and is also faster than 
prior FPGA testing methods. We present 1- and 2-diagnosable built-in self-tester (BISTer) 
designs that make up the ROTE, and that avoid expensive adaptive diagnosis. To the best 
of our knowledge, this is the first time that a BISTer design with diagnosability greater than 
one has been developed for FPGAs. We also develo ... 

Keywords: FPGAs, built-in self-tester (BISTer), diagnosability, functional testing, on-line 
testing, roving tester (ROTE) 



54 Advanced test solutions: On path-based learning and its applications in delay test and Q 
diagnosis 

Li-C. Wang, T. M. Mak, Kwang-Ting Cheng, Magdy S. Abadir 

June 2004 Proceedings of the 41st annual conference on Design automation 

Full text available: ^ pdf(447.54 KB ) Additional Information: full citation , abstract , references , index terms 

This paper describes the innplennentation of a novel path-based learning methodology that 
can be applied for two purposes: (1) In a pre-silicon simulation environment, path-based 
learning can be used to produce a fast and approximate simulator for statistical timing 
simulation. (2) In post-silicon phase, path-based learning can be used as a vehicle to derive 
critical paths based on the pass/fail behavior observed from the test chips. Our path-based 
learning methodology consists of four major compo ... 

Keywords: delay test, machine learning, statistical timing simulation 



55 Tools and strategies for dynamic verification: Industrial experience with test generation Q 
lang ua g es for processor verification 

Michael Behm, John Ludden, Yossi Lichtenstein, Michal Rimon, Michael Vinov 
June 2004 Proceedings of the 41st annual conference on Design automation 




Additional Information: full citation , abstract , references , index terms 



http://portal.acm.org/results.cfm?query=%22con'elation%20between%20tram^ 11/28/04 



Results (page 3): "correlation between training" and test 



Page 5 of 6 



Full text available: ^ pdf (91.99 KB) Additional Information: full citation , abstract , references , index tenms 

We report on our experience with a new test generation language for processor verification. 
The verification of two superscalar multiprocessors is described and we show the ease of 
expressing complex verification tasks. The cost and benefit are demonstrated: training 
takes up to six months; the simulation time required for a desired level of coverage has 
decreased by a factor of twenty; the number of escape bugs has been reduced. 

Keywords: functional verification, processor verification, test generation 



Protocol confornnance test generation using multiple UIO sequences with overlap ping Q 
B, Yang, H. Ural 

August 1990 ACM SIGCOMM Computer Communication Review , Proceedings of the 
ACM symposium on Communications architectures & protocols, volume 20 
Issue 4 
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This paper describes an optimization nnetliod for reducing the length of protocol 
confornnance test sequences by overlapping test subsequences obtained using UIO 
sequences. It is shown that test sequences generated by this method are substantially 
shorter than those generated by other methods employing UIO sequences. 

5^ Pa pers: TSNLP: Test Suites for Natural Language Processing Q 
Sabine Lehmann, Stephan Oepen, Sylvie Regnier-Prost, Klaus Netter, Veronika Lux, Judith 
Klein, Kirsten Falkedal, Frederik Fouvry, Dominique Estiva!, Eva Dauphin, Herve Compagnion, 
Judith Baur, Lorna Balkan, Doug Arnold 

August 1996 Proceedings of the 16th conference on Computational linguistics - Volume 
2 

Full text available: ^ pdf(677.10 KB ) Additional Information: full citation , abstract , references , citings 

The growing language technology industry needs measurement tools to allow researchers, 
engineers, managers, and customers to track development, evaluate and assure quality, 
and assess suitability for a variety of applications.The TSNLP (Test Suites for Natural 
Language Processing) project has investigated various aspects of the construction, 
maintenance and application of systematic test suites as diagnostic and evaluation tools for 
NLP applications. The paper summarizes the motivation and main ... 

58 Re gular papers: More accurate tests for the statistical significance of result differences Q 
Alexander Yeh 

July 2000 Proceedings of the 17th conference on Computational linguistics - Volume 2 

Full text available: ^ pdf(665.32 KB ) Additional Information: full citation , abstract , references 

Statistical significance testing of differences in values of metrics like recall, precision and 
balanced F-score is a necessary part of empirical natural language processing. 
Unfortunately, we find in a set of experiments that many commonly used tests often 
underestimate the significance and so are less likely to detect differences that exist 
between different techniques. This underestimation comes from an independence 
assumption that is often violated. We point out some useful tests that do not ... 

59 A built-in test language for PROLOG to validate knowledge-based systems Q 
F. Belli, I. Camara, A. Schmidt 

June 1990 Proceedings of the third international conference on Industrial and 

engineering applications of artificial intelligence and expert systems - 
Volume 2 

Full text available: pdf(886.21 KB) Additional Information: full citation , abstract , references , index terms 
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This paper refines and completes the results of our previous work represented in /3/. Our 
present work concentrates on the innplementatlon of a uniform, white-box test environment 
PROTest (PROLOG Test Environment). PROTest supports the development of object- 
oriented, rule and knowledge-based expert systems which will be implemented in PROLOG. 
PROTest assists the programmer - as well as the quality engineer - in particular to generate 
test cases, to exercise ... 

60 Poster session 2: A new test pattern generator for high defect coverage in a BIST 
environment 
C. Laoudias, D. Nikolos 

April 2004 Proceedins of the 14th ACM Great Lakes symposium on VLSI 

Full text available: ^ pdf(21 1 .40 KB) Additional Information: full citation , abstract , references , index terms 

In tliis paper we propose a new Test Pattern Generator (TPG) for the detection of realistic 
faults occurring in CMOS nanometer teclinologies. The proposed TPG compares favorably to 
the corresponding already known TPGs with respect to the fault coverage obtained by test 
sequences of the same length. Another advantage of the proposed TPG is that the same 
TPG can be used for testing more than one modules in a SOC. 

Keywords: built-in self test, path delay fault, stuck-at fault 
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